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ABSTRACT 



We consider Pj~ T policy of a dam in which the water input is an inverse 
Gaussian process. The release rate of the water is changed from to M and 
from M to (M > 0) at the moments when the water level up crosses levels A 
and down crosses level r (r < A), respectively. We determine the resolvent of the 
dam content and compute the total discounted as well as the long-run-average 
cost. We also find the stationary distribution of the dam content. 



I. INTRODUCTION 

Adel-Hameed (2000) discuss the optimal control of a dam using P* f T policies, 
using total discounted cost as well as the long -run average cost. He assumes that 
the water input is a compound Poisson process with a positive drift. The release 
rate is zero until the water reaches level A, then it is released at rate M until it 
reaches level (r < A), once the water reaches level r the release rate remains zero 
until level A is reached again and the cycle is repeated. At any time , the release 
rate can be increased from zero to M with a starting cost K\M , or decreased 
from M to zero with a closing cost K^M . Moreover, for each unit of output, 
a reward R is received. Furthermore, there is a penalty cost which accrues at a 
rate g, where g is a bounded measurable function on the state space. When the 
release rate is zero, the water content is denoted by I = (It). Let W\ be the 
first passage time of the process I through the boundary A. When the release 
rate is M the content process is denoted by I* = (i" t * ) , the process I* is a strong 
Markov process. Let W* be the first passage of process I* to the boundary 
r. Over time, the content process is obtained by hitching independent copies 
of the processes I and I* together. The content process is best described by 
the bivariate process B = (Z,R), where Z — (Z t ),R = (Rt) describe the dam 
content, and the release rate respectively. We define the following sequence of 
stopping times : 

T = inf{i > : Z t > A}, f „ = inf{f > T Q : Z t = r} 

T„ = m£{t > T n _! : Z t > A}, T n = M{t > T n : Z t = r}, for n>l. 
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It follows that the process B is a delayed regenerative process with the regen- 

* 

eration points being the T n , n = 0,1,... . The penalty cost rate function is 
defined as follows 



g(z) (z,r)e(0,A)x{0} 
g*{z) (z, r) e (r, oo) x {M} 



where g : (0, A) :— > R + , g* : (r, oo) :— > R + are bounded measurable functions. 

For any process Y = (Y t , t > 0}, and functional f, E y (f) denotes the expecta- 
tion of / conditional on Y — y, and P y (A) denotes the corresponding probabil- 
ity measure. Throughout, we let R + = [0,oo), N = {1,2,...}, N + = {0,1,...}, 
and I a ( ) be the indicator function of any set A. Let Cg(0, x, A), be the ex- 
pected discounted penalty costs during the interval (0,W\) starting at x, and 
Cg»(M,y,r) be the expected discounted penalty costs , starting at y, during 
the interval (0, W*) . Furthermore, let C g (0, x, A), C g * (M, y, r) be the expected 
non-discounted penalty costs during the same intervals, respectively. It follows 
that 

Cf(0,x,\) = E x e- at g(I t )dt, C^{M lV ,r) = E y e- at g{I* t )dt, 
Jo Jo 

Q,(0,x,A) - E T g{I t )dt, C g *(M,y,T) = E v g*{P t )dt 
Jo Jo 



Bae et al. (2003) consider the average cost case of the above model, when the 
dam has a finite capacity. Abdel-Hameed and Nakhi (2006) treat the case where 
the water input is a diffusion process. The release rate depends on the water 
content. Dohi et al. (1995) consider the case where the water must be released 
at a fixed time T. They assume that the input is a wiener process. In this 
paper, we discuss the case where the water input is an inverse Gaussian process. 
The assumption that the water input is of that type is more realistic than the 
cases where it is assumed that it is a Wiener process. This is true because the 
inverse Gaussian process has increasing sample paths. We determine the total 
discounted as well as the long -run average costs. In Section II, we discuss the 
resolvent operators of the processes of interest. In Section III we obtain formulas 
for the cost functional using the total discounted as well as the long-run average 
cost criteria. 



II. THE DAM CONTENT PROCESS AND ITS 
CHARACTERISTICS 

Assume that the water input in the dam I = {I t , t > 0} is an inverse Gaussian 
process, with parameters fi, a 2 > . We let p(t, x, y ) be the probability 
transition function of the process I given J = x, and p(t, y ) be its probability 
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transition function given 7 = 0. Since the process I is additive, we have that 
for y > and x > 



P(t,x,y) = p(t,y-x) 



= ) ^(^ eXp{ " 1 *^ } ' 2/> - t (1) 
{ ,y<x. 

It is known that I is a pure-jump process, with state space (0,oo),and with 
jump measure v concentrated on (0,oo) given by 

V ( dv ) 1 cxp(-y M 2 /2a 2 ) 

It follows that for a > 

Ee~ aI * = e-TzW^^+^-t*} 

Direct differentiation of the above function gives, EX t = — , and Var(X t ) = 

To evaluate the cost functional and other parameters of the process during 
the first part of a cycle, we define the Levy process killed at A, as follows: 

X = {I u t<W x } 

Let U a be the resolvent operator of the process /, defined for every bounded 

oo 

function / by U a f(x) = J f(x + y)U a {dy). Then U a is the unique solution of 

o 

the equation 

oo oo 

J f(x + y)U a (dy)=E x J e~ at f{X t )dt. 



It follows with fp(x) = exp(~(3x), (3 > 0, that 

Throughout we let ip z (.) as the standard normal density function, erf() and 
erf c() be the well known error and complimentary error functions, respectively. 
Inverting the above function w.r.t (3 we have 



U a (dy) = —y z {^/a)dy + {^^)e<*^-rtevic{^—^}dy (2). 
= u a (y)dy 
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where 



u a {y) = ^=<p z (Vvi*M + (*L_^) e «*(*£-*0 erf c {^y^=^}. 

It follows that, for x < A, 

C«(0,x,A) = [ X g(x + y)U a (dy), 
Jo 

C g (0,x,X) = f X g(x + y)U (dy). 
Jo 

From Equation (8) of [7], it follows that for x < A, 



E x (exp(-aW\)) = aU a I [X:00 )(x). 



aa 2 — 2fi 



(3) 



where the last equation follows by integrating U a (dy) over the interval [A, oo) (we 
omit the proof). 

It follows that, for x < A, the distribution function of W\ (denoted by Fw x ()) 
is given by 



FwAt) = jW (A -;^-* } - erfc{ (A -^ +t }, * > 0. 

Furthermore, for x < A 



<-\l~)dy. 



E X (W X ) = U I [0 ,x)( x ) 

= + rVA^TV, (VA^g) + (A - y + ^ erf (/^g) (4) 

where the last equation follows from the equation before last upon tedious cal- 
culations which we omit. 



4 



To derive C°. (M, y, r), C g , (M, y, r), E y (exp{-aW*)), and £j,(W r *). We 
first note that / * is a Levy process and using Doob's optional sampling theorem 
we have the following result 

E x [e- aW *] = e -(*-r)i?(a) _ ( 5 ) 
where rj(a) is the unique increasing solution of the equation 

Mr)(a) = a + — .(6) 

It can be seen that the permissible solution of this equation is (we omit the 
proof) 



_ a (1 - Mfi) + ^2aMa 2 + (1 - M^f 
^ ~M + MV • 

It follows that, for any x > r, 

f 1 if fj,M > 1 

Pr{VK* < OO} = < -2(*-t)(1- m M) 

* [ e S?^ if fiM < 1. 

It is also found that the probability density function of W* (fw*(-)) is equal 
to zero for t < ^jp-, and for t > ^jf^-, 



( ) a^{Mt-{x-r)f Pi 2(Mt - (x - r))a 2 j " 



Furthermore, 



= oo if fiM < 1, 



and 



,„,,, (x-t)M 2 (j 2 ., 

= fcM-l)3 lf MM > lj 
= oo if /iM < 1. 

We now define, the killed process 

X* = {I t ,t < W*} 

It can be shown that the process X* is a strong Markov process. Furthermore, 
it has state space (r, oo). Starting at x e (r, oo), let f(x, y, t) be the transition 
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probability function of /. Let U* be the resolvent operator of the process X* , 
it follows that for x £ (r, oo), 

K( d y - x ) = \P* a (V - x ) ~ exp(-(z - T)ri{a)) P * a {y - r)]dy. 
where for any z , 

/•OO 

(z) = / exp(-at)p(t, z + Mt)dt, 
Jo 

where p(t, x) is the transition probability of the process /, starting at zero, 
defined before. 

Furthermore, for x, y > r, 



1 

f(x, y, t) = p{t, y-x + Mt)- Jp(t-s,y-T + M(t - s))f w . {s)ds. 



Thus, for x > r, 

g*(y)U*(dy-x), 

and 

C g ,(M,x,r)= / g*(y)US(dy-x). 



Now we need to compute the joint distribution function of the pair ( W\ , Iw x ) given 
Iq = 0, denoted by fo(t,x). We define v a (x) ~ u a (x)I{x > A} and = 
u a (x)I{x = A}. For any function g we let £/3(g) be its Laplace transform with 
respect to (3. From Equation (8) P. 2067 of [7] we have, for a > 0, (3 > 



Lp(v a ). 



= aL/) (u Q ) H — (u Q ) . 

= aLp(v a ) + — ===== L fj (v a ) -^L fj (v a ) 

V2/3er 2 + ^ 2 - M CT 

= aL p (v a ) + 2L p (u * v°) + +2L l3 (u )L /3 (v a ) - —L p (v a ) 
= aLp(v a ) + 2L (u o *v°) + +2Lp(u o *v'j-^Lf3(v a ). (8) 
Inverting the above function with respect to a,/3, we have 



f {t,x) = ^p(t,x)+2{p t {X)u {x-X)+J u (x-y)-^p(t,y)dy—^p(t,x)}]I{t >0,x>\} (9) 
To find the marginal pdf of Iw x , we define 

L,(X) d M Eo (e-^). 
Letting a I in Equation (8) we get 

L p (X) = 2[L p (u Q * v°) + L fj (u * v' ) - -^Lp(v )}. 

It follows that 



EMw x ) = UoI ^ 0) (10) 

E X {W X ) 
M 

Furthermore, the pdf of I\y x , given I = (denoted by fi w ()) is given by 
fiw, ( x ) = 2[u (A)u (x — A) + / M (a; - y)u' (y)dy ^u (x)}]I{x > A}. 



III. THE TOTAL DISCOUNTED AND LONG RUN- AVERAGE 
COSTS AND THE STATIONARY DISTRIBUTION OF THE 

DAM CONTENT 

We now discuss the computations of the cost functional using the total dis- 
counted cost as well as the long-run average cost. Let W be the length of the 

first cycle, i.e., W=T\ — Tq, and C a (x) be the expected cost during the interval 

* 

[0,T ), when Z = x. Then, it follows that the total discounted cost associated 
with an Pyf T policy is given by 

n (\ ^ n i ^(exp(-ar )£; T C , a (l) 

C a (X,T) = C a (x) H — — — — 

1 - E T (exp(-aW)) 

where C Q (1) is the total discounted cost during the interval (0, W). For x < A, 
we have 
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C a (x) = M{K 2 + K\E x (e~ aWx ) — RE x f ° e - at dt} 



+ / e- at g{Z u R t )dt 
Jo 

= M{K 2 + K x E x {e- aWx ) - RE x [ ° e _Q 'dt} 

/•W A /.To 

+25./ e- at g(I t )dt + El e~ at g*{I* t )dt 

Jo Jff A 

= Af{x 2 + K 1 ^( e - QM/ -)-i?£; x { e - QW ^ /Wx / e - at rft} 

Jo 

/■w T 

+C«(0,T,\)+E x {e- aW >E IWx / e- at g*(i;)dt) 

Jo 

= M{K 2 + K!E x (e- aW >)} + Cg(0, r, A) + ^[ e - a ^C s Q ,_ flM (M,7 WA ,r)], (11) 

where the third equation above follows from the second equation upon condi- 
tioning on the sigma algebra generated by (W\, Iw x ) -Furthermore, 

E T C a (l) = C a (r), 

where C a (r) is obtained from Equation (10) upon substituting r for x. 



Throughout the remainder of this section we 
for x < A 



E x (e~ a ^) = E x [e- aW *E IWx (e- a ^)} 

= E x [e~ aWx e~ r '( a ^ Iw >-~ T ' >v ^] 
= E [e~ aWx -*e~ ri( ° a)( - Iwx -* +x ~ T) ] 

= (a + ip{r,{a))e-^ a ^ x -^ f e - zrl(a) U a (dz) 

J [A — x.oo) 

= M^aje-"*"'^- 7 ' f e- z ' n{a) U a {dz), (12) 

J [A— x,oo) 

* 

where the first equation follows since for x < A, T = W\ + W* and upon 
conditioning on the sigma algebra generated by (W\, I\v x ), the second equation 
follows from Equation (5) above, the fifth equation follows from Equation (8) of 
reference [7] and the last equation follows from Equation (6) above. However, 
for x > A, we have 
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E x (e- a *°) 



E{e- aW ^) 

e -n(a)(x-T). 



Furthermore, 



E T (e- aW ) = E T (e- a ^°) 

= Mr,{a) f e- xv{a) U a (dx), 

J[X-T,00) 

where the last equation follows from Equation (12) above. 

Now we turn our attention to computing the cost functional for the long- run 
cost average case. It follows by a Tauberian theorem that the long run average 
cost per unit of time, denoted by C(A,r) is given by 

M\K - RE T (E Iwx (W*))} + E T [C r (M, Iw x , r)] + C fl (0, A, r) 
C(X,r) = . (13) 

Note that E T (W) = E T W X + E T E IWx (W*) = oo, if M[i < 1. However, if 
Mfj, > 1, we have 



E T {W) = E T {W X ) + E T E IWx (W;) 



= E (W X - T ) + 
= Eo(W X - T ) + 

= E (W X _ T ) + 



(jiM - 1) 
E (W X _ T ) 



(jiM - 1) 
LiME (W X - T ) 



(fiM - 1) 



(14) 



the second equation follows from Equation (7) above, the fourth equation follows 
from Equation (10) above and E (W X - T ) is given in Equation (4) above. 

Now we turn our attention to finding the stationary distribution of the con- 
tent process. It follows that, when M/j, > 1, the content process is ergodic. We 
let Z = lim Z t and F(z) be the distribution function of the process Z. For 

t >oo 

simplicity of the notation, we now denote Uq and Uq by U and U* , respectively. 
It follows from Equations (13) and (14) above that 
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F(z) 



(M»-1)[C° [o J0,t,\)+E [C^ z] (M,I w ,_ t +t,t) 

MfiE (W x _ T ) 
(M M - l)Ui [0 ^ z _ T) (Q) + e [u* [q z _ t) (I Wx )} 



MtiE (W x _ T ) 



• (15) 
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